Fine-grain placement and viewing of virtual objects in wide-area augmented reality environments

ABSTRACT

Provided herein are exemplary embodiments directed to systems and methods of receiving a reference origin, the reference origin including a first longitude, a first latitude and a first altitude above sea level, receiving a point of interest, the point of interest including a second longitude, a second latitude and a second altitude above sea level, and receiving an updated reference origin, the updated reference origin comprising a third longitude, a third latitude and a third altitude above sea level. Further exemplary embodiments include associating digital data with the point of interest.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of co-pending U.S. application Ser.No. 15/600,449, filed May 19, 2017, which claims the benefit of U.S.Provisional Patent Application Ser. Nos. 62/340,110, filed May 23, 2016,and 62/340,118, filed May 23, 2016, all of which are incorporated byreference herein.

FIELD OF INVENTION

Embodiments of the disclosure relate to the placement of digital data inaugmented reality environments.

SUMMARY

According to some exemplary embodiments, methods include receiving areference origin, the reference origin including a first longitude, afirst latitude and a first altitude above sea level, receiving a pointof interest, the point of interest including a second longitude, asecond latitude and a second altitude above sea level, and receiving anupdated reference origin, the updated reference origin comprising athird longitude, a third latitude and a third altitude above sea level.Further exemplary methods include associating digital data with thepoint of interest, adding a 3D offset to the point of interest todetermine a location for the point of interest, determining a threshold,the threshold representing a predetermined change in distance of thereference origin, updating the reference origin when the threshold ismet or exceeded, and/or receiving an initial location, the initiallocation including GPS coordinates of a client device, an altitude abovesea level of the client device and an offset from the altitude above sealevel of the client device.

Various exemplary methods include creating by the client device avirtual East North Up coordinate system based on the initial location,receiving a current course location of the client device, the currentcourse location including new GPS coordinates, and receiving digitaldata associated with an offset. Additional methods include creating bythe client device a virtual East North Up coordinate system based on theinitial location, the current course location including a new altitudeabove sea level of the client device, recording the current courselocation and the new altitude above sea level on the client device,inputting the current course location and the new altitude above sealevel of the client device into a filter, and/or fusing the filteredinformation with data from sensors to estimate a position in East NorthUp coordinates.

Other exemplary methods include receiving an initial location, receivinga current course location, determining a difference between the initiallocation and the current course location, receiving acceleration dataand GPS speed, filtering the acceleration data, the GPS speed and thedifference with a filter, and outputting improved location coordinates.Other exemplary methods include filtering latitude, longitude, andaltitude data with the filter.

Certain exemplary methods include receiving location data, the locationdata including GPS coordinates, formatting the location data for afilter, inputting the location data into the filter, and filtering bythe filter the location data and altitude data. Additional exemplarymethods include obtaining from a sensor on a client device additionaldata for filtering by the filter.

Exemplary systems include a computing device, the computing devicefurther comprising a GPS receiver, a sensor, a network communicationmeans communicatively coupled to the GPS receiver, the sensor and anetwork, and a filter communicatively coupled to the computing device.The filter may include a data converter.

Further exemplary methods may include viewing through a viewport on aclient device a first 3D digital media object placed in an augmentedreality space, the first 3D digital media object remaining in a fixedposition within the augmented reality space while moving the clientdevice in a real world setting. Additional exemplary methods includereceiving an initial location, the initial location including GPScoordinates of the client device, an altitude above sea level of theclient device and an offset from the altitude above sea level of theclient device, creating by the client device a virtual linear coordinatereference system based on the initial location, receiving a currentcourse location of the client device, the current course locationincluding new GPS coordinates, and receiving digital data associatedwith an offset. Other exemplary methods may include the moving of theclient device in the real world setting including moving around thefirst 3D digital media object that remains in the fixed position and/orwherein the first 3D digital media object was placed in the augmentedreality space by a third party. The first 3D digital media object may beplaced in the augmented reality space by a user of the client device,the user of the client device may place a second 3D digital media objectrelative to the first 3D digital media object, and/or a third party mayplace a second 3D digital media object relative to the first 3D digitalmedia object.

DESCRIPTION OF THE DRAWINGS

Certain embodiments of the present technology are illustrated by theaccompanying figures. It will be understood that the figures are notnecessarily to scale. It will be understood that the technology is notnecessarily limited to the particular embodiments illustrated herein.

FIG. 1 is an exemplary example illustrating a user navigating the worldusing an AR device and placing digital media in that world.

FIG. 2 is an illustration of an exemplary user view from a client devicerunning digital media software where by the user has placed a singledigital media object in an AR environment.

FIG. 3 is an illustration of an exemplary user view from a client devicerunning digital media software where by the user has placed multipledigital media objects in an AR environment.

FIG. 4 is an illustration of an exemplary user view from a client devicerunning digital media software where by the user has placed a Media Tagin an AR environment.

FIG. 5 is an illustration of an exemplary process for calculating realworld locations and converting them into virtual ENU computer graphicspace coordinates for precise placement of digital media in an ARenvironment.

FIG. 6 is a flowchart of an exemplary process that digital mediasoftware running on a client device may perform for calculating andconverting real world locations into virtual ENU computer graphic spacecoordinates for precise placement of digital media in an AR environment.

FIG. 7 is a flowchart of an exemplary process that digital mediasoftware running on a client device may perform to obtain an accuratealtitude for the user's device at a given GPS location.

FIG. 8A is an illustration of an exemplary process for calculating theorientation of digital media being placed in an AR or VR environment.

FIG. 8B is an illustration of an exemplary process for calculating thenormal of digital media being placed in an AR or VR environment. Thisnormal may then be used to store the orientation of digital media.

FIG. 9 is a block diagram of an exemplary system for filtering GPS andacceleration data from a client device to provide the most accurate andprecise location possible, given a variety of inputs and constraints.

FIG. 10 is a block diagram of an exemplary example showing thecomponents and processes that may be involved in implementing a filterand/or fusion system.

FIG. 11 is an illustration of an exemplary example of a user viewingdigital content through their client device which has digital mediasoftware utilizing various exemplary embodiments to view that object asintended by its placement author.

FIG. 12 is a flow chart of an exemplary architecture depicting thephysical system components for practicing aspects of the presenttechnology.

DETAILED DESCRIPTION

Given the two-dimensional nature of existing social media content andthe current limits of existing immersive AR and VR systems, thecreation, sharing, and viewing of classic social media content is notpossible within AR and VR 3D computer-generated environments.

Current social media platforms are unable to precisely determine thelocations of users because users lose context for their posts since onlycoarse-grain places can be associated. For example, “I am currentlysomewhere in Centennial Park” versus “I am currently sitting on the parkbench that is closest to The Pavilion at Centennial Park”.

In terms of existing AR and VR solutions, these systems typically havelimits including, but not limited to:

inaccurate location-based anchoring which utilises a GPS receiver toattain a current coordinate in the world.

inaccurate real-world anchoring of graphics content (i.e. content shakeor jitter).

low-precision location of graphics content (e.g. positional fluctuationof +/−10 metres from session to session).

What is needed is a mechanism that allows for precise and accurateplacement of digital media (including social media information) in adigital AR or VR environment. Such a mechanism should allow users toaccurately, consistently, and repeatedly view that digital contentacross a variety of users and devices given variations in both technicaland environmental variables.

In order to meet these goals, the mechanism should strive to achieve thefollowing:

maintain stable and precise localisation and orientation without helpermarkers (i.e. attain a user's position in the world to an accuracy of+/−1 centimetre and with an accuracy of +/−0.001 degrees in someexemplary embodiments).

obtain an acceptable level of “motion anchoring” (i.e. achieving visualstability of all computer graphics content relative to their perceivedlocation in the real world).

A method for allowing users to place, locate, and view virtual digitalmedia objects (“digital media”) in AR environments at greater accuracythan is attainable by using GPS coordinates alone. This method alsoenables the reciprocal ability for a viewer to view those digital mediainstances from arbitrary viewpoints such that the digital media appearsat the same location, and optionally orientation, as the creatorintended. A viewing user could be the author that placed the digitalmedia or another user. Digital media may be placed within a computerworld graphics coordinate system that has its origin located at a singleparent GPS (latitude/longitude/altitude) coordinate). Certain aspects ofthis method also apply to VR worlds.

The various exemplary embodiments herein solve the problem whereby avirtual content author (i.e. a user) has only a single current GPSlocation but wishes to place virtual digital media objects around thatreal-world location. For example, even if the current GPS accuracy is towithin 10 metres, this method would allow for the placement of one ormore virtual digital media at any discreet distances relative to thatGPS location. Furthermore, the virtual digital media should be perceivedas persisting in the same AR space, relative to visual real-worldreference points, as the original placement was intended by that user.This solution achieves this across a variety of devices and various realworld environmental variables. It also allows for the accurate placementand consistent viewing of multiple digital media that have been placedin close proximity to each other with the intention to appear as asingle object as illustrated in FIG. 3.

To achieve these results, various exemplary embodiments can:

track a user's movements precisely and smoothly in the real world.

allow users with different equipment to accurately solve for the sameposition accurately and repeatedly.

eliminate drift in the positions of digital media that might otherwiseoccur if simple raw GPS data were to be used to identify locations.

provide sub-meter accuracy from software running on a variety of deviceswhere raw GPS measurements are typically within accuracies of 3+m(1-sigma) in horizontal and 10+m vertical.

achieve consistent results across variables including:

different GPS sensors that have different chipsets, noise profiles, datarates, and accuracies and exposure of helpful satellite data (e.g. GNSSraw data).

different sensors such as, but not limited to: accelerometers andbarometers that may produce different data or datasets, and have varyingavailability and update rates.

satellite constellations that vary over time (e.g. 12 hour orbit, not afixed pattern) and in varying availability in satellite quantities.

effects of weather (local and atmospheric) on GPS data.

effects from urban, land canyon, and foliage

different operating systems and versions offering variable locationsupport and precision.

Some aspects of exemplary embodiments include:

Obtaining an initial reference position at which to:

anchor digital media relative to.

view existing digital media relative to.

Performing fine grain motion tracking over time, to predict the mostaccurate location given a variety of variables including, but notlimited to: loss of GPS signal, infrequency of GPS signal, and suddenchanges in environment or movement.

Calculating the most accurate location where digital media is placed.

These are discussed in further detail throughout this specification.

Additional Notes:

Example coordinates provided in the figures within this patentspecification are for demonstration purposes only and don't necessarilyreflect values that may be encountered or calculated in the real world.

Various exemplary embodiments described herein play a key role in theplacement and viewing of Media Tags in AR and VR environments. For moreinformation see U.S. Non-Provisional Application No. ______ filed on May18, 2017, titled “Media Tags Location-Anchored Digital Media forAugmented Reality and Virtual Reality Environments.”

Assumptions and Terminology

Augmented Reality (“AR”): a digital system or interface through whichthe user can view their surroundings with augmentations of that view.Any discussion of AR or related aspects of it refers to augmentation ofa real world or “AR” environment. An AR platform creates a virtualgraphics coordinate space that coincides with the real-world spacearound a user and renders computer graphics relative to that virtualgraphics coordinate system such that those graphics appear to exist inthe real world. An appropriate viewing device is also assumed for theuser, such as but not limited to: a head-mounted display (e.g. augmentedreality glasses or goggles) or a smart phone (i.e. acting as a viewingportal that displays computer graphics on top of, or blended with, alive video feed of the world as seen by camera hardware embedded in thedevice).

Virtual Reality (“VR”): a virtual reality platform creates a virtualgraphics coordinate space into which computer graphic content isrendered in such a way that when viewed through a viewing device, allthe user sees are computer graphics. No real world objects are seen inthis environment. Appropriate viewing and interaction devices are alsoassumed for the user, such as, but not limited to: head-mounteddisplays, optionally with body or body-part motion tracking sensors andsoftware, smart phones (i.e. acting as a viewing portal that displayscomputer graphics on top of, or blended with, a computer graphicsenvironment background).

Accuracy: in a geographical context, accuracy is how close we canidentify a location to its actual real location. For example, a user maybe located halfway between addresses 41 Main Street and 42 Main Streetin reality, but due to limitations in accuracy, mapping software mayonly be able identify the user as being in the 4000 block of MainStreet.

Precision: in a geographical context, precision is how fine one canpinpoint a location on a map. In terms of GPS coordinates, generallymore decimal places in the latitude and longitude components offer moreprecision. For example, as an analogy, placing a pin into a paper map toidentify a location is much more precise than circling that locationwith pen.

Filter: a method such as a mathematical process that takes one or moreinputs and produces an output. In the context of various exemplaryembodiments, a filter may be used to convert data from multiple inputs(e.g. GPS sensors) and sources (e.g. historical data) to a more accurateform. A filter may optionally take input from one or more filters and/orsend its output to one or more filters.

Fusion: the process of using data from multiple sources to produce anoutput (e.g. using GPS and acceleration data to derive a very preciselocation). Fusion may involve the usage of one or more mathematicalfilters as well as historical data to analyse the input and produce amore accurate output.

Media Tag or Tag: a digital media element that a user can place at alocation to attach information to that location. See the [Media Tagpatent spec] for additional information.

Client Device: a device used by a user to place and view digital contentin an AR or VR environment.

Digital Media Software: software running a client device which providesthe functionality to place and view content in an AR or VR environment.

Viewport: the field of view provided by the client device.

AltitudeASL or ASL: altitude above sea level. A common measurement ofelevation relative to sea level.

Callback: in software engineering, a callback is a function or methodregistered for invocation by another entity (e.g. the OS) so that thesoftware can be notified about an event (e.g. a particular method to becalled automatically when the device senses it has changed location).

Application Programming Interface (“API”): in software engineering, anAPI is a set of functions, methods, or other interface componentsthrough which software entities can exchange digital information.

Instantaneous Location: a location obtained by using only the most up todate location information available on the client device. Note that thiscould include usage of historical data.

Continuous Location: the process of collecting location information overtime and using that information to improve the accuracy and reliabilityof future location information.

East North Up (“ENU”): a Cartesian space coordinate system in which aposition is represented using coordinates along axis' corresponding toeast (X) and north (Z), while the “up” (Y) axis indicates the elevation.In the context of this patent specification, an ENU coordinate systemrefers to the computer graphics coordinate space whose origin is locatedin the real world at a specified GPS location. ENU is well suited tovarious exemplary embodiments herein because it provides a locationwhich can be correlated to a real world GPS position, while the “up”component can store an elevation.

Land Canyon: environmental elements such as buildings which limit theline of sight angle(s) between a client device and GPS satellites in thesky.

Initerial Measurement Unit (“IMU”): a sensor capable of measuring realworld data (e.g. an accelerometer or gyroscope).

Geocoordinate: a coordinate representing a real world location,typically comprised of latitude, longitude, and altitude.

Computer vision techniques: methods which identify specific patterns asviewed through a client device, which are then replaced with computergraphics content. Such techniques represent an alternative approach forplacing digital media in an AR world.

FIG. 1 provides an exemplary overview diagram 100 of a user 105 with aclient device running digital media software 110. Note that allcoordinate values in diagram 100 are contrived for demonstrationpurposes and don't necessarily reflect real values. The user's initialcourse location 115 consists of GPS coordinates 43.21N, 78.12W, and anASL of 120 m, and is recorded by the digital media software on theclient device 110 as the reference origin. This reference origin will beused as the location that the placement of digital media will berelative to (details on this placement are provided in the descriptionbelow corresponding to FIG. 5).

Determining an initial location may involve a process to derive the mostaccurate location possible. For example, it may be necessary to wait fora “stable” location by analysing a number of location updates providedby the client device 110 and then running a process to determine whichof those locations is the most accurate. This can be achieved using avariety of methods, such as, but not limited to, the following set ofsteps:

cleaning up the GPS coordinates received from the client device.

ignoring outlier positions (i.e. ignoring locations that lie outside therange of the majority of coordinates).

ignoring glitches in GPS data (e.g. sudden spikes in coordinates due tonoise).

ensuring the chronological order of GPS data is correct (e.g. sometimeGPS data may arrive out of order (older before newer).

averaging GPS locations to ensure the user is not moving around, or toeliminate those related to user movement.

The user then walks a certain distance away 120 from their initialcourse location 115, then stops and points their client device 110 at apoint of interest 135. They then take a picture of this object ofinterest, record a video of it, and/or enter a text comment, and placedigital media 140 at the point of interest. The location for that pointmay be calculated as the user's current course location 125, plus a 3Doffset 155 from that location. Current course location 125 consists ofGPS coordinates 43.012N, 79.12W, and an ASL of 120 m. The orientation ofthe digital media 140 may also be calculated and stored so that italways appears in the same orientation as when it was placed, whenviewed in the future or by other users. Additional details oncalculating orientation are provided below in the descriptioncorresponding to FIG. 8A, and FIG. 8B.

The user then continues to walk in another direction 145. Once they passa certain threshold distance (e.g. 2 km) from their initial courselocation, the digital media software on the client device 110 may thenrecord that position as a new initial course location to use as areference point. Since the Earth is not a perfect ellipsoid, updatingthe reference origin every so often such that locations for digitalmedia placement are proximal to their reference origin, helps to preventthe increasing mathematical error in calculating the placement locationthat would occur if the reference origin was never updated.

FIG. 2 provides an exemplary example of digital media software on aclient device 110 which is displaying a computer graphic doll 200 thathas been placed in an AR environment at a precise location.

FIG. 3 provides an exemplary example of digital media software on aclient device 110 which is displaying multiple digital media objects 300that have been placed into AR environment to appear as if they form asingle object. This example further exemplifies the importance ofprecise and accurate location calculations for objects. Although theobjects 300 in FIG. 3 are separate objects, they have been placed insuch a way that collectively, they appear as a single object constructedof multiple shapes. Thus precision placement and orientation arecritical to ensuring that the objects 300 appear as they were originallyplaced, regardless of the client device being used to view them. Otherfactors such as, but not limited to, the elimination of GPS and sensordrift, are also critical for ensuring correct placement, particularlyfor multiple objects that must appear as one like those in FIG. 3.

FIG. 4 provides an exemplary example of digital media software on aclient device 110 which is displaying a Media Tag 400 that has beenplaced in AR environment at a precise location. For additionalinformation about Media Tags see U.S. Non-Provisional Application No.______ filed on May 18, 2017, titled “Media Tags Location-AnchoredDigital Media for Augmented Reality and Virtual Reality Environments.”

FIG. 5 provides an exemplary conceptual example 500 with greater detailabout the placement process. In example 500, a user 505 is navigatingand then placing a Media Tag in an AR environment. The user 505 startsan “initial location” 515 determined to be at the GPS coordinates oftheir client device 510 and at a height calculated as the ASL 535 plusan offset 520 from that ASL. The purpose of offset 520 is to allow forthe calculation of the device's ASL because the typical altitudeinformation obtained represents a ground level ASL, where as user's aretypically using (e.g. holding) their device while standing. This offsetmay be user configurable or fixed. An example of the latter might be tochoose a “typical” height at which a person holds their client device510 above the ground, so that the digital media software running on theclient device 510 always assumes that height above ground level.

This real world location is then recorded as the user's initial location515 and the digital media software running on the client device 510records this as such. It also creates a virtual ENU coordinate system545 where by the origin of this ENU coordinate system 545 is located inthe real world at the initial location.

The purpose is to represent real world spherical (i.e. Earth)coordinates which are indexed by latitude, longitude, and altitude, intoa three axis linear reference system, typically a Cartesian coordinatesystem that can represent a position in a computer graphics environment.While ENU is one linear reference coordinate system that can be used,any suitable linear coordinate system may alternatively be used such asbut not limited to, one which represents west on the X axis, south onthe Z axis, and up on the Y axis, for example.

The user 505 then navigates in the world arriving at a new GPS location530, possibly with a change in ASL. This location is referred to astheir “current course location”. The user chooses to place digital media550 (e.g. a Media Tag) at some offset 540 relative to them when standingat their current course location 530, based on the yaw, pitch, and/orroll of their client device 510. The purpose of offset 540 is to allowthe user to specify the placement of digital media relative to theirlocation. This is particularly useful when a user is unable tophysically position themself close or near to the point of interest thatthey wish to add digital media to.

Note that logic may optionally be included in the digital media softwareto assist the user in the placement of digital media such that the usercan see a reasonable level of detail when placing the digital media. Forexample, the digital software may enforce a minimal proximity to theuser to prevent them from placing the digital media too close whichwould obfuscate the viewport of the user's client device 510, resultingin a poor user experience.

The offset in front of the user 540 may consist of user configurable orfixed distance value(s) provided by the system. For a user configurableoffset, the user may be provided an appropriate user interface withinthe digital media software running on the client device 510, allowingthem to visually place this location. For example, a user may beprovided with an icon on their client device's touch screen display,allowing them to “nudge” the offset in front of them.

The current course location 530 is recorded by the digital mediasoftware on the client device 510 as the GPS location of the clientdevice 510, as well as its height, which is calculated as ASL 535 plusoffset 555. This location may then be input into a filter (e.g. Kalmanfilter) and optionally fused with data from other sensors such as, butnot limited to: accelerometer, barometer, and historical GPS data, toestimate a true, accurate position in ENU coordinate space 545.

The calculation of the true position in ENU coordinate space 545 can beperformed using a variety of methods including, but not limited to, thefollowing formula:

Position X=(Current Longitude−Current Longitude)*111319.4917* Cos(OriginLatitude*0.0174532925)

Position Y=Current AltitudeASL−Origin AltitudeASL

Position Z=−(Current Latitude−Origin Latitude)*111133.3333

where by:

The current longitude and latitude and current AltitudeASL are theuser's GPS location and altitude which are to be converted to ENU space.

The origin longitude and latitude and the origin AltitudeASL are the GPSlocation and altitude of the reference origin.

The Position (X, Y, and Z) calculated by this formula is the position inENU coordinate space.

The Earth's circumference through poles is assumed to be 40008000 m andthe Earth's circumference at the equator is 40075017 m.

The digital media software on the client device 510 provides anappropriate user interface on its viewport to show and possibly allowfor user adjustment of the offset 540 in front of the user at which thedigital media will be placed. Once placement at that location isconfirmed by the user, the digital media software on the client device510 then stores this offset from the user's current location, and theuser's current location itself.

The heading, yaw, pitch, and optionally roll of the client device 510may also be captured by the digital media software on the client deviceto record the orientation of the digital media being placed. These canbe used to control the azimuth and vertical position of the digitalmedia. Further details about this are provided below in the descriptionscorresponding to FIG. 8A and FIG. 8B.

The client software now has everything it needs to serialize for thedigital media so that it may be accurately placed by an author andviewed by other users. This includes, but is not be limited to:

The location (GPS and altitude) of where the user was when they placedthe digital media in the world (more specifically that of the clientdevice 510).

The world offset of the digital media within the ENU coordinate system545, whereby the ENU coordinate system's origin is located at the user'sinitial location 515. The world offset is calculated by adding offset540 to the user's location 530 to provide an offset relative to the ENUcoordinate system's origin. Since the world offset is relative to ENUcoordinate system's origin, its value may be relatively small (e.g. fromwithin a few meters to 100 meters). Hence the importance of continuallyupdating the initial location as the user navigates over widerdistances, so that the numbers used in calculating these world offsetsremain fairly small and thus reduce or eliminate any loss of precisionwhich would have been encountered with very long distance world offsets.More generally these world offsets are “anchored” to a known location,such that the placement of multiple objects (e.g. those in example 300)can be accurately reproduced in a viewport to ensure they look the sameas when they were placed.

(Optional) The orientation of the digital media. May be used by thedigital media software to reproduce the orientation of the digital mediawhen viewing it. This may also be important when placing multipleobjects to appear as one (e.g. those in example 300).

Note that the process for anchoring the ENU coordinate space at a GPSlocation, and more generally, converting GPS coordinates to ENUcoordinates may only apply to AR environments. VR environments alreadyexist in Cartesian based computer graphics coordinate spaces andgenerally have no GPS coordinates to convert from.

Note: for additional information about serialization and other aspectsof the digital media software on the client device 510, see U.S.Non-Provisional Application No. ______ filed on May 18, 2017, titled“Media Tags Location-Anchored Digital Media for Augmented Reality andVirtual Reality Environments.”

FIG. 6 illustrates an exemplary process 600 that digital media softwaremight use for calculating placement of digital media in an ARenvironment, such as in example 500.

The software is started on the client device 605 which checks for theuser's initial course location. This location may come from a locationupdate event 610 (e.g. a callback within the digital media softwarewhich is invoked from the client device's operating system) which mayexpose location service data 615 that is available on or to the clientdevice. The location service data 615 may include data from a variety ofsensors on the client device 605 such as, but not limited to: GPSsensor, accelerometer, and barometer. The location service data 615 maybe frequently updated by the software and/or client device, causingsubsequent location update events 610 to occur.

Location service data 615 which may originate from the client device,may consist of data from multiple sources that has already been fused bythe client device or operating system such as, but not limited to:information from GPS satellites, WiFi, and/or cellular towers. Thus node610 and other components of a system like example 600 may need to beprogrammed to work with raw or fused data. Note that fused data from theclient device may provide a quick way for digital media software toobtain a course location while the device is obtaining a satellite fix.

The location update event 610 may optionally perform fusion frommultiple types of data such as, but not limited to: GPS coordinates, GPSspeed, altitude from a lookup service, and acceleration. The locationupdate event 610 may also optionally perform filtering on the data. Theoutput from location update event 610 is an accurate location in ENUcoordinate space. Note that the digital media software may convert thisoutput back to geo coordinates (e.g. latitude, longitude, and altitude)for storage purposes.

Once a location event 610 has provided a location, the GPS coordinatesare recorded by the software as the user's current course location 620.The user's altitude 625 is then determined using an appropriate method(see the description below corresponding to FIG. 7 for additionalinformation on determining height). The GPS coordinates of the currentlocation 620 along with the altitude 625 are then converted to ENU space630. If an initial course location has not already been obtained, thenan initial course location will be determined and thus will form thelocation for the origin of ENU coordinate space 630. The initial coursemay be determined using a variety of methods to account for issues suchas, but not limited to, GPS satellite stabilization. An example of sucha method is to take a number of locations provided by location updateevent 610 and from those, choose an average location weighted byaccuracy as the initial course location. The location may be stored bythe digital media software as a geocoordinate. If a previous locationalready exists, then the distance between the current location and theinitial course location is calculated in sub process 635.

A check 640 is performed to see if this distance is greater than somethreshold or if an initial course location needs to be determined. Ifeither condition is satisfied in check 640, then the GPS coordinates andaltitude for the current course location are stored as the referencelocation 645 where the origin of the virtual ENU coordinate system willbe placed at, in which case the GPS coordinates and altitude for thisreference location 645 are cached 650 by the software. If bothconditions for check 640 are not satisfied, then the GPS and altitudefor the current location are converted 655 to a coordinate in thevirtual ENU coordinate space, after which the software caches this ENUcoordinate 660.

Using an appropriate user interface, a user may provide input 670resulting in event 675 indicating the user's intention to add or modifydigital media at their current location. The digital media software thencalculates a placement offset 680 (3D vector) in ENU coordinate spacebased on the direction in which user's client device is facing (e.g. aposition two meters in front of the user's client device, which the useris looking at through their device's viewport). The distance in front ofthe user may consist of user configurable or fixed distance value(s)provided by the system. The calculation 680 may also factor in the yaw,pitch, and/or roll of the user device's orientation.

The software then calculates the final placement coordinate 690 in thevirtual ENU coordinate space, which is relative to the origin of thiscoordinate space. It may do so by adding placement offset 685 to thecached current location 660 to determine placement coordinate 690.

Final placement of the digital media 695 may involve caching numerouspieces of data such as, but not limited to: the GPS coordinates/altitudeof the cached reference location 650, the virtual ENU coordinate spacecoordinate 690, and optionally, the orientation at which to place thedigital media (not shown in flowchart). The digital media software mayoptionally store data to the server such as, but not limited to: thegeocoordinates of the authoring user, the ENU space offset to thedigital media, an orientation vector for the digital media, andinformation related to the digital media itself (e.g. comments, images,etc.). The software can now accurately and consistently display thedigital media at the placement location in the real world. Such data maythen be used to reconstruct the digital media object at the exactlocation where it was placed (e.g. when another user views it throughtheir client device with digital media software).

FIG. 7 illustrates an exemplary process 700 for calculating a user'sheight/elevation. Calculating accurate elevation is essential forperforming accurate placement of digital media in an AR environment.However, altitudes derived from GPS are well known to have a best caseaccuracy of about 1.5 metres—three times worse in accuracy than theirhorizontal position accuracy. In addition, online lookup services at aparticular latitude/longitude may have inconsistent levels of accuracydue to the multitude of technologies used to initially obtain the data.Therefore a procedure may be required to help achieve a more accuratealtitude measurement for a given location such as that illustrated inprocedure 700.

The digital media software is started 705, and the current GPScoordinates are captured 710 from the client device. When starting, thedigital media software may calibrate the client device's compass toensure it provides the most accurate location data possible. As the usercontinues to use the client device, periodic location update events 712(e.g. callbacks in the digital media software invoked from the clientdevice's OS) may occur to provide updated GPS locations using locationservice data 714 available on or to the client device.

The latitude and longitude 720 of the current location 710 are theninput to a ground height look up process 726, which may utilize anexternal ground height lookup service 728 (e.g. communicating with a webbased ground height lookup service through an API) to obtain the knownelevation at that GPS location based on known geographical data. Notethat the digital media software may utilize one or more external groundheight lookup services 728 for various reasons including, but notlimited to: availability at a given location or choosing a service withthe required level of accuracy.

The latitude and longitude 718 of the current location 710 are alsoinput into a process that looks up the mean sea level offset 724. Forexample, input may be with respect to the WGS84 datum's ellipsoid forwhich node 724 converts the offset to a more accurate form such as anEGM96 datum. Offsets to translate altitude into the EGM96 datum may bepulled from a table stored in the client device's memory or from alookup service of some sort.

The GPS altitude 716 is input into a process 722 that calculates therunning mean of GPS altitude. An elevation is then calculated 736 bysubtracting the output of the sea level lookup 724 from the running meanGPS altitude 722. This is input into a process 744 which calculates afinal, relatively precise elevation by fusing elevations from multiplesources.

The ground height look up process 726 may or may not be able obtain aground height value for a variety of reasons such as, but not limitedto, availability of Internet connectivity.

If check 730 indicates that ground height look up process 726 was ableto obtain a ground height, the value is then input into a process 740which calculates the device's elevation (altitude) by taking the groundheight value from process 726 and adding to it, the device height aboveground 742. The device height above ground 742 may be derived using avariety of methods such as, but not limited to, simply using a fixedvalue above ground which represents a typical average height at whichusers hold their client devices. The result of the calculation performedin process 740 is the absolute altitude at which the user's device iscurrently located, which is then input into process 744. If check 730indicates that ground height look up process 726 was not able to obtaina ground height, then process 740 will not be invoked and therefore subprocess 744 will not receive input from process 740.

The software may perform check 732 to see if the client device has abarometer from which an altitude for the client device may be obtained.If check 732 indicates the availability of a barometer, then a processfor capturing the barometric altitude 734 is invoked by the software andthe resulting barometric altitude 738 is then input into sub process744. Since a barometer my not be calibrated, process 734 may need toperform this task. There are a variety of methods of calibrating abarometer such as, but not limited to: using the device's currentlocation to locate the nearest weather stations andinterpolate/extrapolate a mean sea level atmospheric pressure at thecurrent location.

Sub process 744 uses elevations calculated from one or more sources fromthroughout process 700. An elevation from process 736 will always beavailable, whilst an elevation from process 738 or 740 may or may not beavailable depending on the conditions in checks 730 and 732. If morethan one elevation input is available, process 744 “fuses” these inputsto provide the most accurate elevation possible. It should be noted thatall three potential elevation inputs to process 744 are altitudes.

Note that process 700 may only apply to AR environments. VR environmentsare generally already in a Cartesian based computer graphics coordinatespace whereby the height of a user is typically generated and known bythe digital media software.

FIG. 8A and FIG. 8B provide exemplary examples 800 and 850 respectively,showing aspects of how the orientation of a user's client device may beused to calculate the orientation of digital media (e.g. a Media Tag)when placing it in an AR or VR environment. Note that this calculationis assumed to be using coordinates from a 3D computer graphics space(i.e. Cartesian coordinates). In the case of AR, this calculation wouldtake place in a virtual ENU graphics space where by the ENU coordinatesof the user's location are known, as are the offset from the user to thelook at point (i.e. this calculation would take place once the user'slocation is converted from real world GPS coordinates and altitude).

Example 800 shows a user 805 located in ENU space 810 at location 10, 0,0. The user 805 is holding a client device 815 (e.g. smart phone) whichprovides a viewport through which they can observe the world. The clientdevice 815 is running digital media software allowing the user to placeand view digital media. Through the viewport of client device 815 theuser is looking at a point of interest 830 and decides to place digitalmedia 825 at that location. The digital media software calculates the 3Dpoint in space in front of the user by taking the user's currentlocation 820 and adding an offset 835 in front of the user at which toplace the digital media. In this example, the final calculated point 830is at ENU coordinates 20, 5, 2, relative to the ENU origin. In otherwords location 830 is the absolute placement position within the virtualENU coordinate space.

Example 850 shows how a normal 890 for the digital media 880 may then becalculated to represent the orientation of digital media. By subtractingthe user's location 855 from the calculated point of interest 870, theresulting vector can then be normalized to form the digital media'snormal. One or more components of this normal may then be adjusted ifnecessary. For example, the normal 870 could be inverted so that its X,Y, and Z coordinates are negated to capture the fact that the digitalmedia is facing towards the user's client device. The final normal 870may then be cached and or stored along with other digital media data.

FIG. 9 is a block diagram of an exemplary example 900 showing nodes thatmay be involved in performing filtering. Nodes 905 through 940 representsome, but not limited to, the types of sensors which may be present on aclient device which can provide data to a filter 950 that can fuse theseinputs together to obtain a more accurate location. By combining andanalysing data in such a manner, filter 950 may be able to best estimatelocations which are more accurate than consuming just GPS and altitudedata alone.

Data which may be input into a filter 950 may include, but is notlimited to:

GPS data 905.

altitude data from multiple sources 910.

the number of GPS satellites 915.

the known accuracy of the latitude and longitude 920.

GPS speed 925.

accelerations 940.

other data 945. Other data may include, but is not limited to:historical data and data from other IMU sensors.

Filter 950 is a system which can be implemented in a variety of ways.Such a filter might make use of one or more mathematical processes or“filters” such as, but not limited to: Kalman filter, high/low passfilter, complementary filter, to produce a more accurate output from thefusion of given inputs.

The output from filter 950 is location information 955 that should bemore accurate than that provided in nodes 905 and 910. Note that filter950 may optionally create and reference its own historical data in theprocess of performing filtering.

The acceleration data 940 and GPS speed 925 allow precise offsets ofmovements between GPS updates to be obtained and provides a moreaccurate absolute location than GPS alone. For example, consider thefollowing events: a second GPS update event received by the digitalmedia software indicates that the user is 10 meters west of that fromthe first GPS update, but with an accuracy of ±5 meters. However, theaccelerometer data received indicates that the user only moved 6 meterswest while the GPS speed data indicates a distance of 7 meters west. Byhaving a filter that can use all of the error profiles of these threesources, filter 950 can now best estimate a user's location and thedegree of probability that that location is the true location of theuser.

The location filter 950 may perform filtering operations on latitude,longitude, and altitude data. For example, a filter can take advantageof barometric altitude and the elevation lookup data to best estimate amore accurate altitude.

Similarly, filter 950 could generate a more accurate orientation. Forexample, a magnetometer (compass) provides an absolute orientation butis affected by local magnetic sources. A gyroscope however, isn'taffected by these local sources and is very accurate; however it givesrotational rates, not an orientation. These rotational rates however canbe integrated to get a noisy, but still helpful change in rotation whichcan be used to smooth out the noise in the magnetometer.

FIG. 10 is a block diagram of an exemplary example 1000 which shows apossible overview of how a filter might be implemented. This filter maybe compiled as part of digital media software or into a standalonelibrary which can be linked or dynamically linked with and used by thedigital media software. A filter may rely on other software componentsand libraries (e.g. open source math libraries).

Filter 1025 performs the fusion process and may work on the premise thatknowing both previous and current velocity, acceleration, and optionallyother data about a user can improve the accuracy in determining thecurrent location. Filter 1025, or one very similar to it, may be usedwhenever the digital media software acquires a new location (e.g. duringcontinuous movement) or when acquiring an initial location on start up.A filter may be composed of a number of sub components and sub filters.In example 1000, filter 1025 consists of all of the sub nodes depictedas being within node 1025.

Capturing a location can take place both when the user is idle (e.g.when they first boot up their client device or remain stationary at alocation), and while they are moving (e.g. they walk a certain distancefrom their previous initial course location). The former case can beconsidered a situation where “instantaneous” location data is required,while the latter may involve “continuous” location tracking to maintainprecision.

Note: filter 1025 and other aspects of digital media software may alsorun when the digital media software is idle (e.g. running in thebackground on the client device). This may be used to continually gatherhistorical data 1055 for future calculations.

Digital media software may also optionally turn off certain featuressuch as, but not limited to, the GPS receiver, when the software isidle. Alternatively digital media software may choose to request whichunderlying sources of data should be used to provide locationinformation. For example, using location data derived from WiFi/cellulartowers may be beneficial in situations such as, but not limited to:

when power savings on the client device are necessary even thoughlocation accuracy may be reduced. This would require that the GPSreceiver on the client device be shut down to conserve battery.

when the user doesn't wish that his precise location be known.

Similarly, the client device and/or its OS may provide fused data fromWiFi/cellular towers in situations such as, but not limited to:

when satellite coverage is poor.

during start up when the GPS hasn't got a good fix yet.

Both instantaneous and continuous movement cases present a number ofchallenges in obtaining a very accurate and precise location. Forexample, current smart phone GPS technology has a typical accuracyrating of ˜3 m for consumer devices which means that the diameter ofinaccuracy potential is ˜6 m (this is a best case with currenttechnology and still only provides around a 68% confidence level ofbeing within this diameter). By employing a filtering process like thatdepicted in example 1000, a digital media system may be able to overcomeproblems and address issues such as, but not limited to:

achieving a more accurate location than that provided by GPS sensor dataalone, both at idle and when moving.

predicting the user's location when GPS data is not available (e.g. whenthere is a lack of Wi-Fi or cellular signal, and/or when the user is outof sight from GPS satellites).

variations in noise and drift across different user devices.

adjustments for noise due to sudden behavioural changes of the user(e.g. the user suddenly speeds up).

variations in the types of GPS data provided on a given user device(e.g. data may include, but is not limited to: position, bearing, speed,or sometimes both).

GPS multipath mitigation (i.e. lessening the impact of errors caused bymultipath GPS signals, typically those caused by canyon effects).

weather conditions.

acceleration offset bias and GPS drift.

differences in data update frequencies between GPS peripherals andaccelerometers on a given user device; typically, GPS systems updatemuch slower than accelerometers. As such, the drift/bias must becontinually integrated between these two data sources that are updatingat different speeds.

adjustments for GPS satellite constellation changes.

GPS Doppler speed resolutions reported by GPS chipsets.

exposure of GNSS raw measurements by the operating system.

bearing and course accuracy fused with accelerometer accuracy.

bounding the accelerometer drift and integration errors from the GPS.

fusing in known dynamics (e.g. people don't accelerate randomly whileusing the application) as pseudo measurements.

smoothing data to overcome hardware biases (e.g. an accelerometer notperfectly aligned with the camera's chassis) along the X, Y, and/or Zaxis.

The hardware components listed in node 1005 are some of the manycomponents that may be available on a client device to provide data tofilter 1025. This could include, but is not limited to:

GPS Receiver 1010—provides raw latitude, longitude, altitude andpossibly other GPS information such as, but not limited to: accuracyinformation and speed.

Cellular/WiFi Radio 1015—provides access to online services such asaltitude lookups.

IMU 1020—other sensors such as, but not limited to, accelerometer etc.

In example 1000, data from GPS receiver 1010 is input into dataconverter A 1035 which converts it into a form expected by the positionKalman filter 1050 (e.g. data converter A 1035 might output WGS84compliant values that the Kalman filter has been programmed to process).The data output from filter 1035 is then input into the position Kalmanfilter 1050 which performs GPS and altitude filtering. Position Kalmanfilter 1050 may cache this information as historical data 1055 and thenread that historical data 1055 when performing future filteringoperations.

The cell/WiFi radio 1015 may be used to provide access to onlineservices such as elevation or location lookups. A communicationssubcomponent 1040 of the filter, may communicate with these servicesthrough the cell/WiFi radio 1015. Data received may be input into dataconverter A 1035 which can convert it into a form expected by theposition Kalman filter 1050.

Node 1020 lists some of the possible IMU sensors that may be present ona client device. Additional data from IMU sensors such as those listedin node 1020 may also be utilized by filter 1025. A separate dataconverter such as data converter B 1045 may be necessary to convert theIMU data into another form expected by a Kalman filter. In example 1000,data converter B 1045 outputs data which is then input into both theposition Kalman filter 1050 and the attitude Kalman filter 1060. Outputfrom data converter B may include data such as, but not limited to:accelerations, magnetometer data, and angular rates.

Attitude Kalman filter 1060 fuses data such as that from anaccelerometer, magnetometer, and gyroscope to produce accurate yaw,pitch, and roll data. Attitude Kalman filter 1060 may save and/or readhistorical data to produce an output.

Filter 1035 then fuses the output from the position Kalman filter 1050and the attitude Kalman filter 1060 to produce a final accuratelocation.

FIG. 11 provides an exemplary example 1100 showing a user 1105navigating an AR world. In example 1100 user 1105 is viewing a 3Ddigital media object 1120 that has been placed in the world, through theviewport 1125 of their client device 1110. The client device 1110 hasdigital media software that utilizes all of the methods providedthroughout this specification to perform this viewing as user 1105navigates.

The virtual object 1120 is perceived as persisting in the same augmentedreality space, relative to visual real-world reference points, as theoriginal placement was intended by the authoring user. The user 1105 iswalking around 1115 the object 1120 and regardless of their position,the object 1120 maintains its position and orientation independent ofthe client device's 1110 position and orientation.

An important aspect is that the object's 1120 position and orientationare solid (i.e. appear to have little or no jitter) when viewed by auser as they move around. This achieved using fine grain motion trackingof the user 1105 over time to determine their location accurately, aswell as the fact that the object 1120 is “anchored” to a real worldlocation (i.e. its ENU coordinate space has an origin at a specific GPSlocation in the real world, and its absolute location is that ENU spaceis relative to that origin).

Note that whilst a digital media object may appear to be solidlyanchored at its origin (i.e. little or no jitter), the digital mediaobject is free to animate and/or move relative to its origin. Suchmotion and animation may or may not be controlled by a user and thismovement operates within its own local coordinate system or relative toanother such as the viewer's. If a digital object is capable of moving,a user can still move around it and view it, and do so with the samequality user experience as that with static digital objects which havelittle or no jitter.

In addition, the ability to reproduce the object's 1120 position andorientation for any user with digital media software, regardless oftheir device type, is accomplished without the use of computer visiontechniques. The various exemplary embodiments herein provide moreaccuracy and precision than that provided by current computer visiontechniques and thus provides a higher quality user experience in termsof faithfully reproducing an object's location and orientation withminimal jitter.

FIG. 12 illustrates an exemplary architecture 1200 depicting thephysical system components that could allow for the placement andstorage of digital media in an AR or VR environment. This architecturemay generally include, but is not limited to, a client device with anAR/VR software application 1220 through which a user interacts with, oneor more servers 1205 for storing data from and communicating with clientdevice 1220, and a computer network 1210 over which client device 1220communicates with the server 1205.

Client device 1220 has software which allows for the creation,placement, and viewing of digital content in an AR/VR world. Clientdevice 1220 may optionally utilize body/motion sensor peripherals 1215and data to perform client-side tasks, and may optionally provide an ARand/or VR user interface 1225 through which a user may interact with.

Examples of body/motion sensor peripherals 1215 may include, but are notlimited to, accelerometers, GPS trackers, and altimeters. Examples ofclient devices 1220 may include, but are not limited to, cellular phones(aka “smart” phones), VR headsets/goggles, etc. Examples of optional ARand/or VR user interfaces 1225 may include, but are not limited to:touch screens, handheld controllers (e.g. game pads), AR glasses thatdisplay a video stream provided by another client device.

The network 1210 may comprise of a communication path consisting of, butnot limited to: the Internet and public and/or private networks. Suchcommunications may take place over wireless and/or wired connections(e.g. Ethernet), using the networking peripherals available on theserver 1205 and client device 1220.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present technology has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the present technology in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the presenttechnology. Exemplary embodiments were chosen and described in order tobest explain the principles of the present technology and its practicalapplication, and to enable others of ordinary skill in the art tounderstand the present technology for various embodiments with variousmodifications as are suited to the particular use contemplated.

Aspects of the present technology are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thepresent technology. It will be understood that each block of theflowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present technology. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

In the description, for purposes of explanation and not limitation,specific details are set forth, such as particular embodiments,procedures, techniques, etc. in order to provide a thoroughunderstanding of the various exemplary embodiments. However, it will beapparent to one skilled in the art that various exemplary embodimentsmay be practiced in other embodiments that depart from these specificdetails.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, theappearances of the phrases “in one embodiment” or “in an embodiment” or“according to one embodiment” (or other phrases having similar import)at various places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments. Furthermore, depending on the context ofdiscussion herein, a singular term may include its plural forms and aplural term may include its singular form. Similarly, a hyphenated term(e.g., “on-demand”) may be occasionally interchangeably used with itsnon-hyphenated version (e.g., “on demand”), a capitalized entry (e.g.,“Software”) may be interchangeably used with its non-capitalized version(e.g., “software”), a plural term may be indicated with or without anapostrophe (e.g., PE's or PEs), and an italicized term (e.g., “N+1”) maybe interchangeably used with its non-italicized version (e.g., “N+1”).Such occasional interchangeable uses shall not be consideredinconsistent with each other.

Also, some embodiments may be described in terms of “means for”performing a task or set of tasks. It will be understood that a “meansfor” may be expressed herein in terms of a structure, such as aprocessor, a memory, an I/O device such as a camera, or combinationsthereof. Alternatively, the “means for” may include an algorithm that isdescriptive of a function or method step, while in yet other embodimentsthe “means for” is expressed in terms of a mathematical formula, prose,or as a flow chart or signal diagram.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

If any disclosures are incorporated herein by reference and suchincorporated disclosures conflict in part and/or in whole with thepresent disclosure, then to the extent of conflict, and/or broaderdisclosure, and/or broader definition of terms, the present disclosurecontrols. If such incorporated disclosures conflict in part and/or inwhole with one another, then to the extent of conflict, the later-dateddisclosure controls.

The terminology used herein can imply direct or indirect, full orpartial, temporary or permanent, immediate or delayed, synchronous orasynchronous, action or inaction. For example, when an element isreferred to as being “on,” “connected” or “coupled” to another element,then the element can be directly on, connected or coupled to the otherelement and/or intervening elements may be present, including indirectand/or direct variants. In contrast, when an element is referred to asbeing “directly connected” or “directly coupled” to another element,there are no intervening elements present. The description herein isillustrative and not restrictive. Many variations of the technology willbecome apparent to those of skill in the art upon review of thisdisclosure.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. The descriptions are not intended to limit the scope of theinvention to the particular forms set forth herein. To the contrary, thepresent descriptions are intended to cover such alternatives,modifications, and equivalents as may be included within the spirit andscope of the invention as defined by the appended claims and otherwiseappreciated by one of ordinary skill in the art. Thus, the breadth andscope of a preferred embodiment should not be limited by any of theabove-described exemplary embodiments.

What is claimed is:
 1. A method comprising: receiving an initiallocation, the initial location including GPS coordinates of a clientdevice, an altitude above sea level of the client device and an offsetfrom the altitude above sea level of the client device; creating by theclient device a virtual linear coordinate reference system based on theinitial location; receiving a current course location of the clientdevice, the current course location including new GPS coordinates;calculating a world offset in the virtual linear coordinate referencesystem using the current course location and relative offset from thecurrent course location, wherein the world offset is relative to theinitial location; receiving a first 3D digital media object associatedwith the initial location and the world offset, the first 3D digitalmedia object located at a fixed position within an augmented realityspace; and, viewing through a viewport on a client device the first 3Ddigital media object placed in the augmented reality space, the first 3Ddigital media object remaining in the fixed position within theaugmented reality space while moving the client device in a real-worldsetting.
 2. The method of claim 1, further comprising the moving of theclient device in the real-world setting including moving around thefirst 3D digital media object that remains in the fixed position.
 3. Themethod of claim 1, wherein the first 3D digital media object was placedin the augmented reality space by a third party.
 4. The method of claim1, wherein the first 3D digital media object was placed in the augmentedreality space by a user of the client device.
 5. The method of claim 3,further comprising a user of the client device placing a second 3Ddigital media object relative to the first 3D digital media object. 6.The method of claim 4, further comprising a third party placing a second3D digital media object relative to the first 3D digital media object.7. The method of claim 1, wherein at least one of the initial locationand the current course location is estimated using the new GPScoordinates and data from at least one additional sensor.
 8. The methodof claim 7, wherein at least one of the initial location and the currentcourse location is estimated by fusing the new GPS coordinates and thedata.
 9. The method of claim 7, wherein the at least one additionalsensor includes any one or more of an accelerometer and a barometer. 10.The method of claim 1, further comprising filtering at least one of theinitial location and the current course location.
 11. The method ofclaim 10, wherein the filtering includes applying a Kalman filter.